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Overview 



Accessible and affordable, community colleges are gateways to postsecondary education, offer- 
ing students new ways to achieve personal and economic goals. However, many students who 
begin courses at community colleges end them prematurely. In an effort to confront this prob- 
lem, the Opening Doors Demonstration is testing the effects of community college programs 
that are designed to increase student persistence and achievement. The programs include vari- 
ous combinations of curricular reform, enhanced student services, and increased financial aid. 

This report describes the background, objectives, and design of MDRC’s evaluation of Opening 
Doors. Six community colleges are participating in the project: Kingsborough Community Col- 
lege (New York), Lorain County Community College and Owens Community College (Ohio), 
Delgado Community College and Louisiana Technical College-West Jefferson (Louisiana), and 
Chaffey College (California). These are mostly large, well-established community colleges that 
offer a range of associate’s degree programs and technical or vocational programs. The six col- 
leges make up four Opening Doors study sites, each implementing a unique intervention: 

1 . Kingsborough: In small learning communities, groups of incoming freshmen take 
classes together and receive vouchers to cover the costs of their books. 

2. The Ohio colleges: New and continuing students who have completed no more 
than 12 credits receive enhanced counseling/guidance and a small scholarship. 

3. The Louisiana colleges: Low-income students who have children under age 18 re- 
ceive a scholarship that is tied to academic performance; ongoing counseling pro- 
vides an opportunity to discuss goals and progress and to arrange for tutoring or 
other help. 

4. Chaffey: Probationary students take a College Success course and receive indi- 
vidualized assistance in reading, writing, or math. 

The Opening Doors evaluation is the first random assignment study of programmatic interven- 
tions in community colleges — making it the most scientifically rigorous test of whether these 
enhanced programs can make a difference. In addition to examining short-term impacts on 
course completion, grades, and certificates or degrees from community college, the evaluation 
will determine whether Opening Doors participants experience longer-term improvements in 
rates of transfer to four-year colleges and universities and in employment, earnings, personal 
and social well-being, health, and civic participation. Finally, the study will provide an in-depth 
investigation into the implementation and cost of Opening Doors programs and into the percep- 
tions and experiences of community college students and faculty in the study sites. A series of 
publications is planned between 2005 and 2009 to inform education policy and practice. 
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Preface 



Community colleges have been in existence in the United States since 1901, but it 
was the publication of the Truman Commission Report in 1947 that gave shape to the 
community college as we know it today. The report called for the widespread establishment 
of affordable public colleges that would serve as cultural centers and offer comprehensive 
educational programs. In the nearly 60 years since the report’s release, the number of 
community colleges in the country has grown to almost 1,200. The colleges award about 
half a million associate’s degrees each year (along with certificates in dozens of high- 
quality occupational areas), and, with their open-admissions policies, convenient locations, 
and lower fees than those of traditional four-year institutions, they are accessible to millions 
of adults who might otherwise lack the preparation or the means to pursue higher education. 

Yet community colleges struggle with a difficult statistic: While almost half of all 
American undergraduate students attend community colleges, the U.S. Department of 
Education has reported that 46 percent of students who begin postsecondary studies at a 
community college do not complete a degree or do not enroll elsewhere within a six-year 
time frame. And, while they are enrolled, many community college students require 
remedial classes in English or math. What can be done to help these students? 

The Opening Doors demonstration attempts to make a difference. MDRC is 
working with six pioneering community colleges that are implementing innovative 
programs to improve curricula and instruction and to offer enhanced student services and 
financial aid supplements, all designed to help students persist in community college and 
earn a credential. For example, Kingsborough Community College, in Brooklyn, New 
York, is implementing a learning-community model in which up to 25 first-semester 
students form a cohort, together taking three “linked classes” (courses, including one 
English course, that are closely integrated in terms of scheduling and content) as a way to 
build a more personalized, supportive learning environment and to improve academic 
performance. Lorain County Community College and Owens Community College in Ohio 
are providing enhanced student services, including more intensive advising, proactive 
counseling, and tutorial support. 

Opening Doors is pathbreaking for another reason: The effectiveness of its 
programs will be evaluated with a random assignment research design, widely considered to 
be the gold standard in determining whether interventions work. In the evaluation, MDRC 
will compare students who receive the Opening Doors programs with a randomly selected 
group of students who do not. The comparison will focus on a broad range of student 
outcomes, including credit accumulation, retention in college, degree attainment, transfer to 
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four-year universities, labor market success, and civic engagement. Because of the random 
assignment design, any differences in outcomes between the two groups of students can be 
attributed with confidence to the Opening Doors programs. MDRC will use the evidence 
from this evaluation to inform higher education policy and to improve institutional practice 
at community colleges throughout the country. 

Other reports — such as Building Learning Communities: Early Findings from the 
Opening Doors Project at Kingsborough Community College, which is being published 
together with this one — will detail the implementation and impacts of Opening Doors at 
six community colleges around the country. This report, however, paints the initiative in 
broad strokes, providing context for the forthcoming studies, and profiling the demographic 
characteristics of the students in the study samples, with the aim of shedding light on ways 
to help more community college students succeed. 



Robert J. Ivry 
Senior Vice President 
Development and External Affairs 
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Chapter 1 

Introduction and Review of the Literature 



If postsecondary education offers the path to economic and personal opportunity, then, 
for many students, community colleges are the gateway. According to the U.S. Department of 
Education, nearly half of all students who begin postsecondary education start at a community 
college. 1 Community colleges serve students in urban, suburban, and rural locations and are lo- 
cated in all 50 states and the District of Columbia. Because of their open admissions policies 
and low cost relative to most four-year institutions, they are accessible to millions of adults who 
might lack the preparation or the financial means to pursue higher education. Community col- 
leges also tend to be geared to serving part-time and working students. 

Despite the accessibility and relative affordability of community colleges, however, 
many students who begin programs at them end their formal education prematurely. Longitudi- 
nal research on postsecondary students indicates that 46 percent of those who begin at commu- 
nity colleges do not complete a degree or do not enroll elsewhere within a six-year time frame. 2 
To be sure, some students who leave community college without completing a degree or trans- 
ferring to another institution never intended to do more than take a few classes, or they soon 
discovered that they were not really committed to attending college. Many others, however, 
strive to earn a college degree, but their efforts are derailed when the competing demands of 
school, work, and family become impossible to meet. While some may lack the basic aptitude 
or skills to perform successfully in college-level courses, others may confront institutional bar- 
riers to persistence, such as inadequate financial aid or lack of access to advisors and mentors 
who will take the time to work with them toward their academic and personal goals. In short, 
some students experience barriers that prevent entry to college, while others feel ignored or un- 
supported after they arrive on campus. 

MDRC launched the Opening Doors demonstration to leam how community colleges 
can implement reforms that help greater numbers of students achieve their academic and career 
goals. Specifically, the demonstration is examining how various programs or interventions that 
represent enhancements to community college instruction, student services, and financial aid 
might affect student persistence and other outcomes, including degree attainment, labor market 
experiences, and personal and social well-being. Opening Doors will measure the effects of 
these enhancements by randomly assigning students who participate in the research to either a 
program group that receives the enhanced services or a comparison group that receives the 



'U.S. Department of Education, 2002a. 
2 U.S. Department of Education, 2002a. 
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standard services offered by the college. By comparing the two groups’ experiences over a pe- 
riod of several years, the evaluation team will be able to measure the difference, or impact, that 
the interventions make on students’ lives, both in the short term and in the long term. 

This report describes the background, objectives, and design of the Opening Doors 
evaluation. It begins below with a brief review of the research on community college students 
and their experiences. Chapter 2 then provides a general description of the six colleges that are 
participating in the demonstration and the four interventions that are being evaluated. Chapter 3 
presents details about the Opening Doors research design, the research questions that will be 
addressed, the types of data that will be gathered, and the strengths and limitations of the 
evaluation. Chapter 4 describes the targeted population and the characteristics of the first group 
of students to enroll in the demonstration. Chapter 5 concludes the report with a summary of the 
publications that MDRC plans to produce over the course of the evaluation. 



A Review of Relevant Research 

There is a considerable body of research on community college students, the benefits of 
a community college education, and why some students persist and graduate while others drop 
out. The following review, which is far from exhaustive, gives an overview of the data and 
theoretical perspectives that have most influenced the design of Opening Doors. 

A Profile of Community College Students 

The American Association of Community Colleges (AACC) reports that 11.6 million 
people are enrolled in nearly 1 ,200 community colleges nationwide. Most of these students are 
older than what is normally considered “college age,” having an average age of 29 years. 3 Mi- 
nority students account for 30 percent of all community college enrollments nationally — which 
is higher than the percentage of minority students attending four-year colleges and universities 
— and 80 percent of community college students balance their studies with full- or part-time 
work. 4 



People decide to enroll in community colleges for a variety of reasons. Data from the 
U.S. Department of Education indicate that nearly 60 percent of the students who enroll in 
community colleges say that their primary goal is to earn a two-year degree or certificate or to 
transfer to a four-year institution; 23 percent state they are mainly striving to obtain particular 
job skills and credentials; and 19 percent are primarily seeking personal enrichment from the 
community college experience. Nonetheless, although it may not be the foremost goal for all 



3 See the AACC Web site: http://www.aacc.nche.edu. 

4 Phillippe and Patton, 1999. 
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community college students, earning a degree or certificate is desired by the vast majority: Only 
16 percent of enrollees say that they have no such expectation. 5 

Despite their hopes of obtaining postsecondary degrees, many who enroll do not con- 
tinue. For example, more than a quarter of all students who enter public, four-year institutions 
do not persist beyond their first year of study. Among students beginning at public, two-year 
institutions, almost half of those enrolled part-time leave without a degree after one year. 
Nearly a fifth of those enrolled full-time depart after a year. 6 A number of personal and situ- 
ational barriers contribute to these low rates, particularly among community college students. It 
has been shown that high percentages of enrollees are not fully prepared for college-level study. 
For example, more than 40 percent of the high school students who, on graduation, enter commu- 
nity colleges have basic reading comprehension skills but cannot understand or evaluate abstract 
concepts or make complex inferences or judgments that require piecing together multiple sources 
of information. Similarly, about 30 percent of graduating high school students can perfonn simple 
arithmetic operations on whole numbers but cannot perfonn basic operations on decimals, frac- 
tions, or roots. 7 In addition, over half of those who enter two-year institutions have two or more 
“risk factors” known to adversely affect the odds of meeting their postsecondary educational 
goals. These factors include, for example, completion of high school by a General Educational 
Development (or GED) certificate, delayed or part-time attendance when first starting college, 
full-time employment when first enrolled, and being or becoming a parent — especially a single 
parent — while enrolled. 8 

The Benefits of a Community College Education 

Ample evidence suggests that the accomplishment of completing a community college 
degree program pays off in the labor market. Data from the Current Population Survey of the 
U.S. Bureau of the Census, for instance, show that, from 1997 to 1999, the annual earnings for 
people between the ages of 25 and 64 who hold an associate’s degree were $33,020, as com- 
pared with annual earnings of $25,909 for those who did not advance beyond a high school di- 
ploma or GED certificate. 9 

Studies of annual earnings that control for gender, race/ethnicity, parental income, and 
other factors more convincingly illustrate significant economic benefits for students who attend 
community college. For example, using data from the National Longitudinal Survey of the High 
School Class of 1972, Kane and Rouse found that those who attended a two-year college earned 

5 U.S. Department of Education, 2003a. 

6 U.S. Department of Education, 2002a. 

7 U.S. Department of Education, 2003a. 

8 U.S. Department of Education, 2002a. 

9 Day and Newburger, 2002. 
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about 10 percent more income than those without any college education. Moreover, their results 
suggest that, for every two semesters of community college successfully completed, earnings 
increased by 4 percent to 6 percent. 10 Grubb’s analysis of the Survey of Income and Program 
Participation (SIPP) from 1984 to 1990 similarly found that individuals who had a community 
college certificate or degree earned more than those with fewer years of education. He also 
found that people who spent two years in a college or university without receiving a credential 
experienced lower economic returns than people who had an associate’s degree. 11 

Because studies show that bachelor’s degrees confer even greater economic benefits 
than two-year degrees — and because a major function of community colleges is to prepare 
students for transfer to four-year institutions — an important question is whether students who 
attend community college later enter and succeed in four-year colleges or universities. Recent 
analysis conducted by the U.S. Department of Education suggests that 25 percent of all students 
who enter a community college subsequently transfer to a four-year institution within five years. 
Among students who begin community college with the stated expectation of earning a bache- 
lor’s degree or higher, the transfer rate is 36 percent within five years. 12 Research suggests that 
community college students who transfer to four-year institutions have similar rates of persis- 
tence as those who start at four-year institutions but that they require more time to finish their 
degrees. 13 Among community college students who transferred to four-year schools, 36 percent 
attained a bachelor’s degree within six years of starting their postsecondary education; by com- 
parison, 5 1 percent of students who started out at four-year colleges or universities completed a 
bachelor’s degree within this time frame. 14 

Although more difficult to document than educational and employment outcomes, there 
are a host of other potential benefits associated with greater educational attainment, and some of 
these may be particularly important in the lives of low-income and minority students who pursue 
postsecondary education in community colleges. First, the college experience can create sources 
of social support and opportunity through relationships with fellow students, faculty, and staff. A 
large body of research suggests that people who enjoy strong and supportive relationships with 
others are better able to handle stressful life events and circumstances and, consequently, to pre- 
serve their emotional and physical well-being. 15 The college experience can prove to be pivotal in 
the lives of many students because it facilitates access to role models and mentors who are 
uniquely qualified to support and guide them toward their personal goals. Studies show that stu- 



I0 Kane and Rouse, 1995. 

11 Grubb, 1995. 

12 U.S. Department of Education, 2001. 
13 U.S. Department of Education, 2001. 
14 U.S. Department of Education, 2002a. 
15 Thoits, 1995; Turner and Turner, 1999. 
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dents who have mentors may have a greater sense of self-worth, be better able to weather personal 
crises, become more aware of educational and career opportunities, and ultimately set higher goals 
for themselves . 16 Finally, to the extent that colleges successfully create a supportive and “bond- 
ing” environment, students — younger students especially — may be less likely to engage in be- 
haviors that compromise their own and the community’s well-being . 17 

Second, longstanding research literature documents a strong positive association between 
education and health. Individuals with higher levels of education have lower mortality and mor- 
bidity rates from many types of disease and display better health behaviors, including lower levels 
of smoking and binge drinking and a lower prevalence of obesity . 18 The relationship between edu- 
cation and health can be seen as part of a more fundamental relationship between socioeconomic 
status (SES) and health, which is especially relevant to any discussion of the circumstances faced 
by low-income populations. The research has consistently identified a “SES health gradient ,” 19 
wherein each incremental step upward in SES — as measured by social class, education, income, 
and occupational prestige — is associated with better health. If the college experience has the di- 
rect effect of exposing people to more and better health infonnation and other resources , 20 as well 
as ultimately providing greater avenues for raising SES through improved educational and em- 
ployment options , 21 then it may play an important role in the long-term health of students, particu- 
larly low-income students who have more ground to gain. 

Finally, greater educational attainment is also believed to promote greater civic en- 
gagement and awareness, as assessed by voting behaviors, volunteerism, newspaper readership, 
and involvement in community groups and clubs (for example, political clubs, youth groups, 
and church service groups). Education is also thought to be associated with fundamental civic 
values and tolerance, such as an ethic of participation in the larger community and attitudes to- 
ward free speech . 22 In a recent analysis of data sets from both the High School and Beyond and 
the General Social Survey, Dee found that education has large effects on voter participation and 
support for free speech. Educational attainment also appears to increase the quality of civic 
knowledge, as measured by the frequency of newspaper readership . 21 Because of their critical 
place in the country’s postsecondary educational system, community colleges may play an im- 
portant role in producing these outcomes. 



16 Rhodes, 2002. 

17 Eccles and Gootman, 2001; Neumark-Sztainer, Story, French, and Resnick, 1997; Resnick et al., 1997. 
T 'hristenson and Johnson, 1995; Deaton and Paxson, 2001; Elo and Preston, 1996. 

19 Adler and Newman, 2002; Marmot and Wilkinson, 1999. 

20 Grossman, 1972; Kenkel, 1991. 

21 Grossman, 1972. 

22 Delli Carpini and Keeter, 1996; Sullivan and Transue, 1999; Uslaner, 2002. 

23 Dee, 2004. 
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Student Persistence and Departure 

Given the foregoing apparent benefits of earning a two- or four-year college degree, re- 
searchers have long sought explanations for why so many students end their studies prema- 
turely. Much of the early research focused on the background and personal characteristics of 
students and their families, such as gender, race/ethnicity, high school performance, the timing 
of college entrance, parental education, and family income. For example, Gates and Creamer 
used the National Longitudinal Survey of the High School Class of 1972 to study retention in 
community colleges, and they built a predictive model that focuses on students’ backgrounds 
and personal characteristics — such as high school grades and the decision to delay college en- 
try — as explanatory factors. That model explained just 4.3 percent of the observed variation in 
community college retention, 24 highlighting the need for broader theoretical frameworks for 
understanding student persistence and completion rates in community colleges. 

Tinto has theorized that three primary sets of factors influence students’ decisions to 
stay in or leave college. In addition to the background and personal characteristics of students 
and their families, he draws attention to students’ interactions with faculty and college staff as 
well as to their relationships with fellow students. Such an “interactionalist” perspective empha- 
sizes the significance of “fitting in” or “feeling at home” on campus, 25 and empirical studies 
have demonstrated the value of such a focus. In one national study that tracked 825 community 
college students for nine years, Pascarella and colleagues concluded that “student-environment 
fit” — which is partly attributable to the frequency and quality of students’ interactions with 
faculty and peers — was the most important determinant of persistence and degree comple- 
tion. 26 Similarly, in a study of 569 students enrolled in the Ford Foundation’s Urban Commu- 
nity Colleges Transfer Opportunities Program, Nora and Rendon found that students who were 
better integrated into the academic and social life of the campus were more likely to transfer to 
four-year institutions. 27 

Recently, scholars have reevaluated Tinto’s theory, attempting to delve deeper into the 
processes through which student commitment increases via integration into the academic and 
social communities of the college or university. 28 These efforts reflect current demographic, cul- 
tural, and organizational perspectives on student engagement and success. A fundamental con- 
tribution of this research — and one that is most relevant to Opening Doors — is that it high- 
lights the role that colleges themselves can play in fostering student persistence and program 
completion. In short, by shifting the focus from what students bring to the college experience to 

24 Gates and Creamer, 1984. 

25 Tinto, 1993. 

26 Pascarella, Smart, and Ethington, 1986. 

27 Nora and Rendon, 1990. 

28 Braxton, 2002. 
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what happens to them after they begin their studies, the field is moving toward a deeper and 
more policy-relevant understanding of the social and institutional factors that contribute to stu- 
dents’ departure from community college. 

Reforming Community Colleges 

Evolving discourse about institutional responsibility for student departure from postsec- 
ondary studies holds a range of perspectives, particularly when the discussion turns to commu- 
nity colleges. Cohen and Brawer, for example, argue that community colleges are already doing 
their best to encourage students who might otherwise have been excluded from higher educa- 
tion and that students who discontinue their studies often do so because of problems or chal- 
lenges that are simply beyond the reach of institutional supports . 29 In contrast, Dougherty con- 
tends that community colleges must consider profound organizational changes — perhaps going 
so far as to convert themselves into branches of the state universities or even into four-year col- 
leges — to address their essential failure to deliver on the educational and occupational oppor- 
tunities that they promise . 30 

There is consensus, however, that community colleges are complex organizations that 
have multiple, and sometimes competing, missions, which include the provision of: 

• Instruction leading to associate’s degrees or transfers to four-year institutions 

• Developmental education for adults who lack the skills required for college- 
level work 

• English as a Second Language (ESL) education 

• Vocational education leading to certification for work in specific industries 

• Noncredit instruction in a wide range of substantive areas 

Every one of these programs has a constituency and may or may not be available at in- 
stitutions other than community colleges. Therefore, while some have suggested that commu- 
nity colleges lack the resources to deliver all the programs that they attempt to provide , 31 it is 
important to recognize that there are often strong political and financial pressures or incentives 
for community colleges to be comprehensive . 32 



29 Cohen and Brawer, 2003. 

30 Dougherty, 1994. 

31 Brint and Karabel, 1989; Cohen and Brawer, 2003; Dougherty, 1994. 
32 Bailey and Morest, 2004. 
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Because more fundamental institutional changes are unlikely in the near term, analysts 
have suggested a range of practical strategies that may improve student persistence in community 
colleges and may help more students realize their academic goals. Matus-Grossman and Gooden 
outline three sets of approaches, focusing on the curriculum and instruction, student support ser- 
vices, and financial aid. 33 With regard to curricular and instructional refomi, for example, colleges 
can integrate developmental education or English language instruction into occupational or aca- 
demic programs to contextualize learning and speed progress toward degrees. Another solution 
that might better accommodate the needs of working and parenting students is to break a single 
credential program into a sequence of modules that may be completed in intensive, short periods. 
Finally, colleges can make scheduling more flexible by offering more classes in the evenings and 
weekends, on-line, in neighborhood centers, or at worksites for major employers. 34 

The educational approaches outlined above are structural in nature, and thus they do lit- 
tle to address what happens inside the classroom. Some studies suggest that community colleges 
pay insufficient attention to teaching quality and that good teaching emerges only in isolated 
and idiosyncratic ways. 35 Developmental English and mathematics courses are particularly 
prone to “skills and drills” models of instruction that rely heavily on completing exercises in 
workbooks or on computers, make little effort to tie basic skills to practical uses in students’ 
lives, and involve little meaningful interaction between students and teachers. As Grubb notes: 

The problem with this approach is not just that these classes are deadly, with 
low levels of student engagement. They also violate all the maxims for teach- 
ing in adult education. . . . And their tactic is simply “more of the same”: they 
take students who have not learned well in 10 or 12 years of standard didac- 
tic instruction, and then put them through an additional 1 5 weeks of similar 
instruction. 36 

Grubb also observes that “skills and drills” teaching is not limited to developmental courses but 
also emerges in college-level classes that are essentially converted into remedial classes when 
students are not ready for what the instructor considers college-level work. 37 

Various proposals have been put forward or implemented to improve teaching and 
learning in community colleges. One approach is to offer supplemental instruction, in which 
students work collaboratively to integrate course content and study skills through regularly 



33 Matus-Grossman and Gooden, 2002. 
34 Kazis and Liebowitz, 2003. 

35 Grubb and Associates, 1999. 

36 Gmbb, 2001a, p. 11. 

37 Grubb and Associates, 1999; Grubb 2001b. 




scheduled, peer-assisted study sessions linked to difficult courses. 38 Another is to create small 
learning communities, in which a group of students takes two or more classes together (often 
including at least one remedial course) and teachers for these classes coordinate their lesson 
plans and review student progress. 39 A more broad-based approach is to give college administra- 
tors and faculty direct feedback from students about their experiences in the classroom — as the 
Community College Survey of Student Engagement is doing — which provides a means for 
promoting campuswide dialogue on academic standards and strategies for improving instruc- 
tion. 40 The Carnegie Foundation for the Advancement of Teaching has also recently launched 
an initiative with 1 1 community colleges in California to design effective models for teaching 
students who come to campus unprepared for college-level work. 

Revamping or strengthening student support services may be another strategy to in- 
crease student persistence and academic achievement. Purnell and Blank identify five distinct 
but interrelated components of a student services program: academic guidance and counseling, 
personal guidance and counseling, career counseling, academic tutoring and supplemental sup- 
ports such as child care and transportation assistance, and vouchers or stipends to cover books 
and supplies. While few colleges can afford to offer these services to all students, the authors 
provide examples of programs that are targeted to low-income or nontraditional students who 
are in greatest need of help. 41 Academic guidance and counseling is arguably the most important 
student service and an area where most students receive minimal help. Nationally, the average 
community college counselor has a caseload of nearly 1,000 students. 42 While there is tremen- 
dous variety in how these counselors deliver services and what topics they cover, large 
caseloads tend to drive them toward a traditional problem-solving approach in which students 
present an issue and the counselor offers a quick response. The National Academic Advising 
Association (NAcAdA) urges community and four-year colleges and universities to provide 
sufficient staffing — so that students and counselors can have ongoing, interactive relationships 
— and to adopt a developmental approach that helps students clarify personal goals and objec- 
tives, rather than simply approving their choice of courses. 43 

Finally, some have suggested that financial aid policies need to be revised in order to 
increase access to, and persistence in, higher education. 44 Rising tuition and fees, coupled with a 
general policy shift toward loans and away from grants, have forced students at all income lev- 



38 University of Missouri-Kansas City, 1997. 

39 Tinto, 1998. 

40 Community College Survey of Student Engagement Web site: www.ccsse.org. 

41 Purnell and Blank, 2004. 

42 Gmbb, 2001b. 

43 Gordon, Habley, and Associates, 2000. 

44 American Association of State Colleges and Universities Web site, 2005; Western Interstate Commis- 
sion for Higher Education, 2003. 



9 




els to take on increasing amounts of debt in order to pursue postsecondary studies. These trends 
may adversely affect community college students, particularly in light of research suggesting 
that many low-income students are “loan averse” — perhaps because they are uncertain that 
their education will “pay off’ in the future and allow them to repay those debts. 45 Further, stu- 
dents who have defaulted on loans in the past may be ineligible for future grants or loans. One 
study found that coming from a low-income family, being Native American or African- 
American, having a GED, having a composite score of less than 16 on the ACT Assessment, or 
being over 25 years old were characteristics associated with high default rates for student loans 
in community colleges. 46 To improve student persistence and accelerate completion of commu- 
nity college degrees and certificates, Choitz and Widom propose four possible solutions: (1) 
providing more intensive financial aid counseling to ensure that students apply for and receive 
all the aid to which they are entitled, (2) more fully covering the direct price of attendance by 
providing more generous grants, (3) encouraging part-time students who are working to in- 
crease their enrollment by supplementing their wages while they are in school, and/or (4) offer- 
ing financial incentives for students to complete key academic milestones or earn good grades. 47 



A Guiding Framework 

While each of the foregoing proposed solutions to increase student persistence and 
achievement in community colleges has merit, few have been subject to rigorous evaluation to 
determine whether they would lead to better outcomes than students would experience within 
existing programs. The Opening Doors study will test this proposition. Based on the preceding 
review of the literature, the general conceptual model shown in Figure 1 . 1 provides a useful 
guiding framework that links community college reforms to outcomes that reflect a successful 
educational experience and transition to a better life. As illustrated in the figure, community 
college reforms theoretically affect educational outcomes like persistence and grades in the 
short term, which, in turn, influence labor market outcomes and other indicators of personal and 
social well-being in the long term. This conceptual model shows that there are many theoretical 
linkages between these concepts, but it highlights the primary relationships — distinguishable 
by the bolded arrows linking community college refonns to short-term outcomes, which subse- 
quently are linked to long-term outcomes — on which the Opening Doors demonstration is 
built. These bolded arrows portray the causal pathway wherein educational gains can serve as a 
lever for more enduring, positive rewards in young adulthood. 



45 Matus-Grossman and Gooden, 2002; U.S. Department of Education, 2002b. 
46 Christman, 2000. 

47 Choitz and Widom, 2003. 



10 




The Opening Doors Demonstration 
Figure 1.1 

Conceptual Model for Evaluating the Effects of Community College Reforms 




NOTE: Bolded arrows illustrate the primary relationships on which Opening Doors is built. 



Other important linkages are shown with unbolded arrows. It is possible, for example, 
that, in addition, some reforms may have direct and independent effects on long-term outcomes, 
such as social networks or civic engagement. Figure 1.1 also demonstrates that the effects of a 
student’s background and preexisting characteristics and circumstances must be considered — 
alongside reforms — as determinants of community-college enrollment and both short- and 
long-term outcomes that occur with the passage of time. Chapter 2 describes the Opening Doors 
colleges and interventions in detail. 
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Chapter 2 

The Opening Doors Colleges and Interventions 



Beginning in 2000, MDRC began a nationwide search for community colleges that 
were both interested in, and had the capacity to participate in, a demonstration project to test the 
effectiveness of programs designed to increase the academic success of their students and that 
would lead to longer-term success in the labor market and in life. Because there was little evi- 
dence to suggest what type of interventions would make the most difference for students, 
MDRC sought out programs that emphasized one or more of the three approaches outlined in 
Chapter 1 : curricular and instructional reforms, enhanced student services, and increased finan- 
cial aid. As noted earlier, the evaluation design that MDRC proposed was to assign students 
randomly into one of two groups: a program group that would receive the Opening Doors in- 
tervention and a comparison group that would receive all the regular programs and services of- 
fered by the college but not the Opening Doors program. A random assignment design is best 
suited to situations where (1) there is clear contrast between the intervention and the status quo; 
(2) there is genuine uncertainty about whether the benefits of the intervention will outweigh its 
costs; and (3) the intervention cannot accommodate everyone who might be eligible, making 
random assignment a fair and ethical way to allocate scarce program slots. 

The nationwide search led to the following six community colleges that were interested 
in participating in the demonstration and that met the conditions necessary for a random as- 
signment study: 

• Kingsborough Community College in Brooklyn, New York 

• Lorain County Community College in Elyria, Ohio (west of Cleveland) 

• Owens Community College in Toledo, Ohio 

• Delgado Community College in New Orleans, Louisiana 

• The West Jefferson campus of the Louisiana Technical College in Harvey, 
Louisiana (a suburb of New Orleans) 

• Chaffey College in Rancho Cucamonga, California (in San Bernardino 
County, east of Los Angeles) 

MDRC worked closely with the colleges to create the interventions that would be 
tested, building on existing programs and goals at each college. MDRC also raised funds to 
provide grants that each college could use to develop or enhance its Opening Doors program. 
This chapter describes the colleges and the interventions that will be tested. 



13 




The Opening Doors Colleges 

This section presents four tables that describe the Opening Doors colleges in terms of 
their location, student body and faculty characteristics, and completion rates (Table 2.1); their 
largest academic and vocational programs (Table 2.2); their costs (tuition and fees) and the pri- 
mary sources of financial assistance for their students (Table 2.3); and their budgets (Table 2.4). 
Comparisons are made both among these six public institutions and between them and their 
counterparts nationally. Data are drawn primarily from the U.S. Department of Education’s In- 
tegrated Postsecondary Education Data System (IPEDS). While at times the discussion focuses 
on the unique characteristics of Lorain and Owens in Ohio and of Delgado and the West Jeffer- 
son campus of Louisiana Technical College, these two pairs of institutions are treated as single 
study sites for the purposes of the impact evaluation. 

Kingsborough, Lorain, Owens, Delgado, and Chaffey are well-established, prototypical 
community colleges that offer a range of programs leading to an associate’s degree or to a tech- 
nical or vocational certificate. In contrast, the West Jefferson campus of Louisiana Technical 
College is newer and smaller and concentrates on providing technical and vocational education; 
the large majority of its students are seeking to acquire job skills and training certificates and 
generally are not pursuing an associate’s degree or the opportunity to transfer to a four-year in- 
stitution. Also, because Lousiana Technical College-West Jefferson is part of a 40-campus, 
statewide system, statistics are not always available to compare this campus with the other 
Opening Doors colleges. 

As Table 2. 1 shows, both urban and suburban institutions are participating in the Open- 
ing Doors evaluation. These are relatively large community colleges that represent a diverse set 
of geographic locales. Kingsborough, Owens, Delgado, and Chaffey are located in or near large 
cities, and each has a full-time equivalent (FTE) enrollment of around 10,000 students. Lorain’s 
student body is smaller, with an enrollment of almost 5,200 FTE students. All five of these col- 
leges are relatively large in comparison to national averages. 1 Louisiana Technical College- 
West Jefferson, however, has an FTE enrollment of only 147 students. 

Community colleges in the United States have more part-time than full-time students, 2 
and, like most of their counterparts nationally, the more traditional community colleges participat- 
ing in Opening Doors serve a greater proportion of part-time students (Table 2.1). This is espe- 
cially true for Owens and Chaffey, each with approximately 70 percent of its students attending on 
a part-time basis. Kingsborough is the exception, with a slim majority of full-time students. At 
Louisiana Technical College-West Jefferson, 61 percent of the students are enrolled full time. 



^hillippe and Patton, 1999. 
2 Phillippe and Patton, 1999. 
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The Opening Doors Demonstration 
Table 2.1 
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White, non- Hispanic (%) 41.9 81.8 81.2 37.6 26.2 33.0 

Race/ethnicity unknown (%) 0.0 1.6 1.4 12.0 0.0 5.3 




Table 2.1 (continued) 
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Graduation rates are for cohort year 2000. Graduation rates are determined by completers within 150 percent of the normal time to complete a degree. 
Louisiana Technical College-West Jefferson calculates graduation rates differently than the other institutions and cannot be compared. 
g Transfer-out rates are for cohort year 2000. Transfer-out rates are determined by students known to have transferred to another postsecondary institution 
within 150 percent of the normal time to complete a degree. 




The student populations at these colleges can be further described in terms of their gen- 
der, race/ethnicity, and age composition (Table 2.1). Generally speaking, community colleges in 
the United States enroll more women than men, although the gender differential is not large. 3 
Among the large community colleges participating in Opening Doors, only Owens reports a 
majority of male students (53 percent). Delgado’s student population has the largest proportion 
of women (69 percent). The student population at Louisiana Technical College-West Jefferson 
comprises significantly more men than women. 

According to national statistics, non-Hispanic whites account for more than 60 percent 
of the country’s community college population. Hispanics and blacks or African-Americans 
each represent sizable minority groups within this population, accounting for approximately 1 1 
percent each. 4 The Opening Doors colleges diverge from this demographic profile in a couple of 
important ways (Table 2.1). Some have exceptionally large non-Hispanic white populations; for 
example, more than 80 percent of students at the two Ohio colleges are predominantly non- 
Hispanic whites. In contrast, other colleges in the study serve larger minority populations: 
Chaffey’s student population is more than 40 percent Hispanic; Kingsborough’s and Delgado’s 
are each more than 30 percent black or African-American; and more than half (56 percent) of 
the student body at Louisiana Technical College-West Jefferson are black or African-American. 

People under age 25 account for close to half the community college student population 
in the United States, and the traditional colleges in Opening Doors generally reflect this. 5 How- 
ever, Kingsborough appears to have an exceptionally young student body, with close to 75 per- 
cent being under age 25. Louisiana Technical College-West Jefferson reaches an older student 
population, with approximately 60 percent of its students being over the age of 25. 

Table 2.1 also presents data on the faculty at the Opening Doors colleges. Because 
IPEDS data do not support the calculation of FTEs for faculty members, the analysis cannot 
include faculty-to-student ratios. What is clear, however, is that, like many of the students they 
teach, most of these faculty members — and their counterparts nationally 6 — have only part- 
time involvements with the colleges. In all these Opening Doors colleges, part-time faculty 
members far outnumber their full-time counterparts, typically by a factor of greater than two. 

The gender composition of the faculty at the Opening Doors colleges is fairly balanced, 
with virtually even splits at two of the colleges and small majorities of women at three others. 
Louisiana Technical College-West Jefferson has a predominantly male faculty. Non-Hispanic 
whites are more prevalent than individuals of color among the faculty of the Opening Doors 



3 Phillippe and Patton, 1999. 

4 Phillippe and Patton, 1999. 

5 Phillippe and Patton, 1999. 

6 Cohen and Brawer, 2003. 



17 




colleges — although, in the Louisiana site, substantial proportions of the faculty are black or 
African-American. At Chaffey, more than 10 percent of the faculty are Hispanic. 

Additionally, Table 2.1 shows that the equated nine-month average salaries of full-time 
faculty members vary considerably across the Opening Doors colleges, which is partly a func- 
tion of regional economic differences and institutional type. For instance, the salaries at Kings- 
borough and Chaffey — located near heavily populated and coastal urban centers — are higher 
than at the others and are substantially higher than the national average for faculty at public 
community colleges. Chaffey has the twelfth-highest faculty salary level among two-year public 
institutions nationally. 7 The nine-month average salary at Louisiana Technical College-West 
Jefferson is the lowest among the Opening Doors sites. 

Table 2.1 also shows how many associate’s degrees and certificates are awarded by 
these colleges, and it presents their graduation and transfer rates. The numbers of degrees and 
certificates conferred in 2002 by the Opening Doors colleges partly reflect institutional size, 
with the largest colleges awarding the most degrees and certificates. They also mirror institu- 
tional type, with the Louisiana Technical College-West Jefferson campus awarding far more 
certificates than associate’s degrees. 

The graduation rates — which reflect the percentage of students who earn a degree 
within 150 percent of the time expected for that degree — vary considerably among the tradi- 
tional community colleges in the study (Table 2. 1). Chaffey has a graduation rate of close to 25 
percent, while Delgado has a rate of just over 2 percent. Readers who are unfamiliar with the 
IPEDS graduation rates should know that they are not necessarily a good measure of institu- 
tional performance, in part because community colleges do not have control over the character- 
istics of students who enroll and in part because many students take longer than three years to 
earn an associate’s degree. 

Many community college students transfer to other schools complete a degree. Lorain 
has a transfer rate of nearly 20 percent — almost 10 percentage points higher than its graduation 
rate — suggesting that many of its students transfer to another postsecondary institution before 
attaining an associate’s degree. Chaffey has the highest transfer rate, at just over 20 percent. The 
graduation and transfer rates for two-year public institutions nationally are 24 percent and 1 5 
percent, respectively. 8 

Table 2.2 presents the most commonly pursued academic and vocational programs of- 
fered by each of the Opening Doors colleges, ranked in descending order for each institution, 
based on the number of graduates in school year 2002-2003. The two most commonly chosen 



7 Based on MDRC calculations using IPEDS data on equated nine-month average salaries for full-time in- 
structional faculty at two-year public institutions during school year 2002-2003. 

Personal correspondence with Andrew Mary, U.S. Department of Education, National Center for Educa- 
tion Statistics, October 21, 2004. The graduation rate is based on cohort year 1999. 



18 




The Opening Doors Demonstration 
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areas of study are in the Health Sciences and Business. These two fields account for the first- or 
second-ranked program across these institutions. Liberal Arts and Sciences is also a highly 
ranked area of study and is the one most frequently sought at Lorain and Chaffey. The rankings 
also highlight some of what distinguishes the Opening Doors colleges from one another. For 
example, programs in Engineering have large enrollments at both Ohio schools. Programs in 
Security and Protective Services are popular at Owens and Chaffey, as are programs in Com- 
puter and Information Sciences at Kingsborough and Lorain. The programs chosen by students 
at the Opening Doors colleges generally mimic the national data for completions of associate’s 
degrees. The five largest programs of study among community colleges nationwide during 
school year 2001-2002 were, in descending order, Liberal Arts and Sciences, Nursing, General 
Studies, Business Administration and Management, and Business (General). 9 

Table 2.3 summarizes infonnation about the costs and sources of financial aid at the 
Opening Doors colleges. The six institutions vary dramatically in this regard. The average stu- 
dent at Chaffey paid just $264 for tuition and fees in school year 2002-2003. In comparison, the 
average student at Kingsborough, Lorain, and Owens paid between $2,000 and $2,800. The cur- 
rent national norm for in-state tuition and fees at two-year public institutions is $1,927. 10 

Table 2.3 also presents data that illustrate the proportions of students receiving various 
forms of financial aid across the Opening Doors colleges. These statistics are indicative of how 
many low-income students these colleges serve, though they also reflect the expense of attend- 
ing a particular institution. Nationally, 57 percent of full-time, first-time students at two-year 
colleges receive some form of financial aid. 11 In comparison, 69 percent of the full-time, incom- 
ing students at Kingsborough — which has the highest costs among the Opening Doors colleges 
— receive some form of grant or loan. This percentage is just over a third at Chaffey and 
Delgado, which have the lowest tuition and fees of the Opening Doors sites. 

The largest sources of student aid at the Opening Doors colleges are federal and state or 
local grants. This is also the case for community college students nationwide, 12 although the 
proportions of full-time students receiving aid vary considerably by location. New York and 
California have large and relatively generous state tuition assistance programs. Thus, for exam- 
ple, Kingsborough reports that 62 percent of its full-time students receive state or local grants, 
while just 2 percent of full-time students at Delgado receive such aid. Institutional grants are 
generally less common, although 15 percent of the students at Lorain receive them. Student 



9 U.S. Department of Education, 2003b. This government report adopts the 1990 version of the Classifica- 
tion of Instructional Programs. 

10 U.S. Department of Education, 2003b. 

U U.S. Department of Education, 2003c. 

12 U.S. Department of Education, 2003c. 
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The Opening Doors Demonstration 
Table 2.3 
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loans are also a prevalent source of financial aid and are more frequently received among stu- 
dents at Owens and Delgado. The average amounts of these grants and loans vary across the 
colleges (Table 2.3). 

Table 2.4 presents budget data for Fiscal Year 2002/03 for the traditional community 
colleges participating in Opening Doors. These are complex institutions with large budgets, 
ranging from over $66 million for Lorain to over $92 million for Kingsborough. Of the tradi- 
tional community colleges, Lorain and Kingsborough have the smallest and largest student bod- 
ies, respectively. For all the colleges, state appropriations are an important source of revenue. 
Local appropriations are significant as well for Chaffey, Lorain, and Kingsborough. Also, tui- 
tion and fees account for sizable portions of total revenue, especially for Owens and Delgado. In 
contrast, tuition and fees account for relatively little of the total revenue at Chaffey, where the 
cost of attendance is exceptionally low. Federal grants and contracts are another significant 
source of revenue for these colleges, particularly at Kingsborough, Lorain, and Delgado. Na- 
tional data suggest that these colleges are generally representative of two-year public institutions 
in the United States, where the largest sources of revenue are state and local appropriations, fol- 
lowed by tuition and fees. 13 

As Table 2.4 shows, the largest category of expenditures for these traditional commu- 
nity colleges is instruction — accounting for between 35 percent and 47 percent of total expen- 
ditures — and these statistics parallel the national average for two-year public institutions (43 
percent). 14 However, the mixture of expenditures is unique from campus to campus, and this 
variation is telling with regard to the institution’s priorities. For example, Lorain devoted 16 
percent of its expenditures to public service, 15 and Delgado appropriated 19 percent of its total 
to scholarships and fellowships. Finally, the colleges’ expenditures per FTE student are sugges- 
tive of their capacity to hire more faculty and staff and build a resource-rich campus. Four of the 
traditional community colleges in Opening Doors are roughly comparable in their expenditures 
per student, ranging from $7,691 (Delgado) to $8,977 (Kingsborough). Lorain stands apart from 
the others, at $ 1 1 ,7 1 6 per FTE student, but this difference may be accounted for partly by the 
fact that the college offers substantial noncredit instruction, customized training, and business 
services to individuals who are not enrolled and thus are not reflected in FTE statistics. Lorain 
also counts some $2.9 million in scholarship funds as part of its total expenditures. 



13 U.S. Department of Education, 2003c. 

14 U.S. Department of Education, 2003c. 

15 Public service includes conferences, institutes, general advisory services, reference bureaus, and similar 
services provided to particular sectors of the community. This function includes expenses for community ser- 
vices, cooperative extension services, and public broadcasting services. 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, Integrated Postsecondary Education Data System (IPEDS). 
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The Opening Doors Interventions 

The Opening Doors demonstration is examining the implementation and effects of the 
four interventions described below. 

Learning Communities and a Book Voucher 

(Kingsborough Community College) 

Since the mid-1990s, Kingsborough Community College in Brooklyn — a branch of 
the City University of New York (CUNY) — has been operating small learning community 
programs for students who require English as a Second Language (ESL) services or who are 
majoring in specific occupational fields. As Tinto describes, learning communities, in their most 
basic form, are a kind of co-registration or block scheduling that enables students to take 
courses together. 16 With the Opening Doors project, Kingsborough has begun offering the learn- 
ing communities to a much larger number of incoming freshmen who are enrolling full time in 
an associate’s degree program. 

The Opening Doors Learning Communities consist of three classes that groups of ap- 
proximately 25 freshmen take together as a block during their first semester. The classes include 
(1) a college orientation course, taught by a counselor, that covers college policies, study skills, 
time management, and other topics; (2) an English course, most often at the developmental level; 
and (3) a standard college course such as sociology, history, or health. The instructors for each 
block are expected to coordinate their syllabi before the semester begins and to meet regularly 
during the semester to discuss student progress. Students in the Opening Doors Learning Com- 
munities also receive vouchers that cover the cost of their books at the student store, and they have 
access to a dedicated tutor who can assist them with English and other course assignments. 

Students in the comparison group are not assigned to the Learning Communities and do 
not take classes that are intentionally linked. Rather, these students take whatever courses are 
available, and they are likely to encounter different students in every class. Comparison group 
students can take the college orientation class and an English course if they wish, but they are 
not required to do so (although most students will eventually need to take English to graduate). 
Comparison group students do not receive book vouchers. Finally, comparison group students 
do not have a dedicated tutor, although they can access tutoring on their own initiative through 
the college’s tutoring labs. 



16 Tinto, 1998. 
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Enhanced Advising Services and a Scholarship 

(Lorain County Community College and Owens Community College) 

Lorain County Community College and Owens Community College are testing a simi- 
lar program that offers students enriched counseling and guidance services and a modest schol- 
arship. Unlike Kingsborough — which is targeting only incoming freshmen who enroll full 
time — the Opening Doors projects at Lorain and Owens are targeted to new and continuing 
students who have completed no more than 12 credits and who have shown indications of aca- 
demic difficulties, as determined by low grades or withdrawal from courses. The program is 
open to students who enroll half time and full time. The linchpin of the program is an advisor 
with whom students are expected to meet at least once a month for two semesters, to discuss 
academic progress and any other issues that may be affecting their schooling. The advisor is 
expected to carry a caseload of no more than 125 students and to be available evenings. In con- 
trast, students in the comparison group see whichever advisor is available and rarely meet with 
an advisor more than once a semester. The regular advising staff do not normally keep weekend 
or evening hours. Moreover, the ratio of counseling staff to students who are not enrolled in 
Opening Doors is about 1 to 1,000. 

To support the work of the Opening Doors advisors and to make them more effective in 
meeting students’ needs, Lorain and Owens have designated staff members from other student 
services departments — including financial aid and career services — to function as a team. 17 In 
practical terms, this means that at least one staff member from each of these departments has 
agreed to serve as a point person for the Opening Doors program and to meet with students who 
are referred by the Opening Doors advisor. Students in the comparison group can access these 
same departments, but they generally do this on their own rather than through referrals. 

Finally, for each of two consecutive semesters, students in the Opening Doors groups at 
Lorain and Owens are given a $150 scholarship that they can use for any purpose. The scholar- 
ship payments are approved by the academic advisor and are made at the beginning and middle 
of the semester, as a way of ensuring that students stay in contact with the advisor. Students in 
the comparison group do not receive these scholarships. 



17 At Owens, the Opening Doors team also includes staff members from the student records and academic 
tutoring departments and from the bursar’s office. 
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A Scholarship Predicated on Academic Performance 

(Delgado Community College and 

Louisiana Technical College-West Jefferson Campus) 

In 2001, Louisiana was one of eight states that began using federal welfare dollars to 
help low-income adults access higher education through the state’s community and technical 
college system. Specifically, the state offered a tuition waiver and child care assistance to low- 
income parents who had a high school diploma or General Educational Development (GED) 
certificate and who met other eligibility requirements. After learning of research findings on the 
positive effects of programs that offered welfare recipients a financial incentive for moving into 
work, 18 state officials became interested in the idea of a similar program that offered a financial 
incentive to low-income parents for completing postsecondary education. 

The Opening Doors project at Delgado Community College 19 and at the Louisiana 
Technical College-West Jefferson campus is offering a $1,000 scholarship to parents with chil- 
dren under age 1 8 whose family incomes are below 200 percent of the federal poverty level. 
Students do not need to be on welfare to qualify. The scholarship is targeted mainly to new stu- 
dents, though current students who are shifting from developmental to college-level courses are 
also eligible. The scholarship is tied to academic perfonnance: An initial payment of $250 is 
made after students enroll at least half time; a second payment of $250 is made after midterms, 
for students who remain enrolled at least half time and earn at least a C average; and a final 
payment of $500 is made after students have passed all their courses. The scholarship is in addi- 
tion to any other financial aid that students receive and is offered for up to two semesters, for a 
total of up to $2,000. Each student is assigned to a counselor and is required to meet with him or 
her at the beginning and middle of each semester to discuss academic goals and progress. The 
counselors are expected to monitor the students’ grades and arrange tutoring or other help as 
needed. The counselor-to-student ratio ranges from approximately 1 to 100 or 150 at Delgado’s 
campuses and from 1 to 75 at the Louisiana Technical College-West Jefferson campus. 

The most obvious difference for students in the comparison group is that they do not re- 
ceive a $1,000 scholarship; nor do they have an assigned counselor who monitors their aca- 
demic perfonnance. The general counseling staff at the Delgado campuses are each responsible 
for 500 to 750 students. The Louisiana Technical College-West Jefferson campus does not em- 
ploy any counselors outside the Opening Doors project. 



ls Bloom and Michalopoulos, 2001; Morris et al., 2001. 

19 Delgado Community College has three campuses: City Park, West Bank, and the Charity School of 
Nursing. Only the City Park and West Bank campuses are participating in the Opening Doors study. 
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Basic Academic Instruction and College Survival Skills 

for Students on Probation (Chaffey College) 

Chaffey College initiated a campuswide Basic Skills Transformation Project in fall 
2000. The primary outcome of this project was to establish writing, math, reading/ESL, and 
multidisciplinary “Success Centers” that offer individualized and small group instruction to stu- 
dents in each of these areas and also provide resources and instructional support to faculty. The 
Success Centers are operated by full-time faculty and a large group of tutors so that students can 
get help either by scheduling an appointment with a staff member or by dropping in from early 
morning through late evening on weekdays and during designated times on weekends. Com- 
puter-assisted instruction and other resource materials are also available in the Success Centers. 
Research by the college suggests that there is a strong, positive correlation between the amount 
of time that students spend in the Success Centers and their academic performance. 

Chaffey’ s Opening Doors program is targeted to probationary students who have at- 
tempted at least 12 credits but have completed no more than 30 credits and either have not com- 
pleted at least half the courses in which they enrolled or have an overall grade point average 
below 2.0. If the students are receiving financial aid, their ability to continue receiving grants 
and loans may be in jeopardy. The Opening Doors program will, for the first time, require pro- 
bationary students to take a College Success lecture course that will teach them skills in time 
management, note-taking and test-taking, critical thinking, and communication. The instructor 
will be a member of the college’s counseling department who will meet with the students indi- 
vidually, assess their academic abilities, and help them create an educational plan that includes a 
weekly requirement to attend one of the college’s Success Centers in the area in which they are 
weakest — reading, writing, or math. An instructor at the Success Centers will be designated as 
a contact person who will monitor students’ progress and help them raise their academic skill 
levels. In addition, in their second semester in Opening Doors, students will take a one-unit 
course to continue the counseling and guidance through the applied learning experience. 

The comparison group will continue to be treated as probationary students are currently 
treated. Specifically, students will receive a letter informing them that they are at risk of dismissal 
from the college and of losing their financial aid (if they are receiving it), and they will be urged to 
attend a Student Success Seminar for probationary students and to see an academic counselor. 
Chaffey ’s large counseling caseloads — a ratio of 1 counselor to approximately 1,500 students — 
make it difficult for most students to receive individualized or intensive assistance from the coun- 
seling office. Comparison group members can enroll in the college’s general guidance course and 
can utilize the Success Centers if they choose, but they will not be required to do so and will not 
be assigned to a specific counselor or Success Center staff member who will work with them over 
time. The college’s experience indicates that most students on probation do not take advantage of 
either the college’s guidance courses or the Success Centers’ resources. 
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Chapter 3 

The Design of the Opening Doors Evaluation 



The Opening Doors sites differ in tenns of which students they are targeting and what 
services they offer, but they have the same objectives: to improve students’ chances of acquiring a 
postsecondary education and earning a degree, with the longer-range objective of helping students 
find rewarding careers and lead productive, satisfying lives. The evaluation team — led by 
MDRC, members of the MacArthur Research Network on the Transitions to Adulthood , 1 and re- 
searchers from Princeton University 2 — will examine how and to what extent students achieve 
these objectives in each site. Specifically, the team will examine the program’s implementation 
(operations and service delivery), its impacts (differences between outcomes for the program 
group and the comparison group), and its costs. The primary evaluation questions follow: 

• How have the colleges designed their Opening Doors programs, and how do 
these programs operate in practice? What are the differences between the 
educational experiences, student services, and financial assistance provided 
to the program group and the comparison group? 

• What are the short-term impacts of Opening Doors on students’ academic 
performance in community college, as measured by certificate or degree 
completions, semester-to-semester persistence, grades, and other outcomes? 

• What are the long-term impacts of Opening Doors on students’ persistence in 
community college or other institutions of higher education? Specifically, 
what are the program impacts on certificate or degree completion and on 
transfer to four-year colleges and universities? 

• What are the long-tenn impacts of Opening Doors on students’ experiences in 
the labor market, social behaviors and networks, health, and civic engagement? 

• What are the costs of running Opening Doors programs, and do the long- 
tenn economic benefits outweigh the costs? 



'The names and affiliations of the members of the MacArthur Research Network on the Transitions to 
Adulthood are listed in the Acknowledgments. For more information about the network, see the Web site: 
http://www.pop.upenn.edu/transad/. 

2 Christina H. Paxson of Princeton University is leading the component of this evaluation that is focused 
on health outcomes. She is also a member of the MacArthur Research Network on Socioeconomic Status 
and Health. 
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The evaluation will address the last two questions only if Opening Doors leads to sig- 
nificant, positive impacts on educational outcomes. This is because advancement in college and 
the attainment of degrees are presumed to be the primary pathways to improvements in job 
quality, wages, and other measures of personal development and well-being. Moreover, benefit- 
cost analysis is worth doing only if the programs lead to positive effects. 

The community colleges participating in the Opening Doors study are responsible for 
identifying students who meet the targeting criteria for their programs (discussed in Chapter 4). 
Eligible students are informed about the study — including random assigmnent to program and 
comparison groups — and are given the option to participate. Students who give their consent are 
asked to complete a baseline form that asks about their educational background and goals, current 
and recent employment experiences, and sociodemographic characteristics. In addition, they are 
asked to complete a short questionnaire that assesses their physical and mental health, including 
their smoking and drinking habits. In appreciation for their time, students receive a $20 gift card 
for use at a local store or the public transit system. Baseline data will be used to describe the sam- 
ple and to identify which subgroups benefit most and least from the Opening Doors interventions 
— for example, younger or older students, or those who begin the study with higher or lower edu- 
cational aspirations. The random assignment to program and comparison groups is perfonned by 
MDRC; afterwards, the colleges are responsible for notifying students of their research status and 
providing them with the appropriate services for their designated group. 



Components of the Evaluation 

The Implementation Study 

The Opening Doors implementation study has several objectives: 

• To describe the institutions where the evaluation is taking place and the in- 
terventions that are being tested 

• To understand the differences in the nature or level of services that are pro- 
vided to the program group and the comparison group 

• To compare and contrast the sites in terms of important institutional and op- 
erational dimensions 

• To document the experiences of students, faculty, and staff in their own words 

All these objectives are in pursuit of a larger goal: to explain how the Opening Doors 
programs are leading to changes in students’ lives. Equally important, the implementation re- 
search will be used to identify lessons that policymakers and college administrators across the 
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country can use in designing and operating better programs to help community college students 
achieve their goals. 

The implementation study will rely principally on qualitative data gathered during field 
visits to each participating college. Specifically, the evaluation team will conduct open-ended 
interviews with college administrators, faculty, and staff throughout the study to leam how the 
college is serving both the program group and the comparison group. In addition, Opening 
Doors classes, counseling sessions, and other program activities will be observed. At selected 
campuses, in-depth qualitative interviews will be conducted with students in the study sample 
— including members from both groups — to gain a deeper understanding of factors on and off 
campus that facilitate or hinder their progress in school. 

The implementation study will incorporate quantitative data as well. Most important are 
the 12- and 36-month surveys, which will include numerous questions for program and com- 
parison group members about their classroom experiences, interactions with college faculty and 
staff, use of college services, and so forth. The surveys will be conducted over the telephone or 
in person, and a small financial incentive will be offered to achieve a minimum response rate of 
80 percent. The 12-month survey has been developed to emphasize questions about students’ 
educational experiences at the community college, both in and out of the classroom and on and 
off campus, and asks about the nature and quality of students’ interactions with faculty, staff, 
and fellow students. 

At selected campuses, the 12- and 36-month surveys may be supplemented by brief 
questionnaires or attendance records for sample members. Data on institutions — including en- 
rollments, student and faculty characteristics, course offerings, revenues, and expenditures — 
will be obtained through the U.S. Department of Education’s Integrated Postsecondary Educa- 
tion Data System (IPEDS). Finally, MDRC is partnering with the Community College Survey 
of Student Engagement (CCSSE) to administer a survey to a randomly selected group of faculty 
at several of the Opening Doors colleges. 3 The faculty survey will provide measures of the in- 
structional climate at the college, including how much and in what ways teachers interact with 
students, the nature of assignments they give, the time they spend teaching and preparing for 
class, and their perceptions of the college’s support for instructional activities. 

The Impact Study 

In order to measure the impacts of Opening Doors programs on individuals, the evalua- 
tion will collect data on outcomes for sample members for at least three years. (If program ef- 



3 Kingsborough, Lorain, Delgado, Louisiana Technical- West Jefferson, and Chaffey community colleges 
have agreed to participate in the faculty survey. 



31 




fects are large and significant — and if sufficient funds are raised — the analysis may be ex- 
tended beyond three years.) In the short term, the evaluation team will look for differences in 
community college course enrollment, student persistence, and degree completion between 
members of the program group and the comparison group. In the long temi, the focus will be on 
program effects on attainment of community college certificates or degrees, transfer to four-year 
colleges and universities, employment and earnings, family structure (including marriage and 
childbearing), peer networks and social connections, health, and civic engagement. 

Two primary data sets will be used to measure the impacts of Opening Doors programs. 
First, MDRC will acquire community college transcripts and other administrative records data 
for all program and comparison group members. The transcripts will be analyzed to determine 
what courses students take, what grades they earn, how many semesters they enroll, and 
whether they complete a certificate or degree. 4 MDRC will also acquire financial aid data from 
the colleges to determine the amount of scholarships and loans received by program and com- 
parison group members. Finally, state unemployment insurance records will be accessed in or- 
der to track sample members’ employment and earnings. 

The second data source for the impact study will comprise the follow-up surveys ad- 
ministered at 12 and 36 months after random assignment. In addition to the questions about stu- 
dents’ educational experiences, the 12-month survey will touch on other aspects of life — em- 
ployment experiences, transfer to other colleges or universities, personal relationships, social 
networks, health, attitudes, and civic behavior — that Opening Doors may influence in the short 
term. The 36-month survey will devote even greater attention to this latter set of issues, as 
changes in the more distal outcomes are not expected until after students have had an opportu- 
nity to complete their community college studies. 

Appendix A describes the types of analyses that will be conducted to determine pro- 
gram impacts. The primary analytic method will use standard statistical tests to determine 
whether average outcomes for program and comparison group members are significantly differ- 
ent. In addition, as detailed in Appendix A, nonexperimental methods may be used to supple- 
ment the experimental analysis. For example, because participation in Opening Doors is volun- 
tary, some members of the program group may not participate in or receive Opening Doors ser- 
vices. Nonexperimental techniques provide a way to estimate program effects on those sample 
members who receive the full Opening Doors treatment. 



4 Although transcripts will be available for every sample member, one limitation is that they will not reflect 
courses taken or degrees earned at institutions other than the current community college. MDRC will rely on 
surveys to capture such information. 



32 




The Cost Study 



MDRC will gather financial data from the colleges to document the overall cost of 
Opening Doors and the cost of individual program components, such as the scholarships and 
counseling services. These figures will be compared with the expenses incurred for each col- 
lege’s standard services, in order to determine the added costs of running Opening Doors. The 
cost data provide a useful measure of the intensity of each site’s interventions and will inform 
policymakers and college administrators who want to expand or replicate Opening Doors pro- 
grams. If the evaluation team detects significant impacts in the Opening Doors sites, it will also 
conduct a benefit-cost analysis to determine whether the financial benefits of the program — 
such as increased earnings for participants, increased tax payments for participants who go to 
work, reduced public assistance benefits, and so on — are greater than the expense of operating 
the interventions. The benefit-cost analysis will take into account the perspectives of participat- 
ing colleges and students, governments and other funding agencies, and society at large. 



Strengths and Limitations of the Evaluation’s Design 

The literature reviewed in Chapter 1 suggests that postsecondary education and training 
programs run by community colleges have the potential to help young adults increase their edu- 
cational attainment, obtain better employment, and achieve other positive outcomes. To date, 
however, there is no hard evidence to back such claims. All the postsecondary studies cited are 
nonexperimental; they reveal only positive associations between college attendance and earn- 
ings or other outcomes. The literature also does not show whether changes in curriculum, stu- 
dent services, and financial aid policies at community colleges help low-income adults who 
have low levels of skills do better than they otherwise would. The use of random assigmnent 
methods in community college research — for the first time, to the authors’ knowledge — will 
demonstrate whether the Opening Doors interventions yield benefits over and above the pro- 
grams and services that the colleges normally provide. The random assigmnent design will also 
enable the evaluation team to determine whether there are causal relationships between com- 
munity college attendance and employment, earnings, or other outcomes (for example, in- 
creased social comiections, improved health status, and greater civic participation). To the ex- 
tent that such impacts are detected, the study will lend considerable force to arguments in favor 
of increased public and private investments in interventions like Opening Doors, specifically, 
and in community colleges, generally. 

Because the interventions that are being tested in Opening Doors were developed with a 
random assignment research design in mind, they attempt to create a sharp contrast between the 
services received by the program group and the comparison group. And because students face 
many different challenges in achieving their academic goals, the interventions also tend to have 
multiple components. Such bundled treatments, however, make it difficult to know whether all 
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the components of the intervention are equally important. For example, if Kingsborough Com- 
munity College’s Learning Communities program (described in Chapter 2) is effective, should 
that be attributed to the block scheduling of courses, the specific inclusion of a college orienta- 
tion course, the book voucher, the tutor, or all the above? 

The implementation and impact studies will shed light on such questions by examining 
whether all the program components are actually delivered as intended and by measuring differ- 
ences between program and comparison group members in actual service receipt or program 
participation. As described in Appendix A, instrumental variables models will be used to esti- 
mate the effect of the “treatment on the treated.” In order to leam precisely the added value of 
each program component, however, the experiment would need to compare alternative interven- 
tions side by side — for instance, randomly assigning students to subgroups that receive block 
scheduling only or block scheduling plus a book voucher. Such a design was considered but 
rejected, out of concern that the colleges could not meet the necessary sample sizes and would 
have difficulty running two interventions simultaneously. Nevertheless, future studies should 
consider such methods to disentangle the effects of individual program components. 

The intensive and generous nature of the Opening Doors interventions — particularly in 
the sites offering scholarships — raises another question: Are these “boutique” programs that 
can benefit only a few students, or can they be operated on a large scale? In most cases, the 
Opening Doors interventions grew out of programs or ideas that college staff had before the 
study began. College administrators in each of the sites have expressed a strong commitment to 
continuing or expanding the programs if the results are positive. In some cases, colleges may be 
able to reallocate existing resources to keep their Opening Doors programs going; in other 
cases, to serve more students, they may be able to tap such external funding sources as Tempo- 
rary Assistance for Needy Families (TANF) and workforce development monies. 

While MDRC has led random assignment studies in a number of institutional contexts, 
community colleges have posed some particular challenges for recruiting a research sample. 
Community colleges often allow students to register in several ways, including coming to the 
admissions office, mailing in applications, or signing up for classes on-line. Such procedures 
make it difficult to provide information about the study to everyone who might be eligible and 
to obtain their written consent to participate, particularly when hundreds (or thousands) of in- 
coming students show up immediately before school starts. Of the Opening Doors colleges that 
have begun random assignment thus far, Kingsborough’ s enrollment process was the most cen- 
tralized — students had to enroll on campus and in person on designated days — which pro- 
vided MDRC and college staff with an efficient means of informing most eligible students 
about the study. The Ohio and Louisiana colleges had no comparable central intake point, 
thereby necessitating the use of phone calls, mailings, flyers, and news media to inform eligible 
students about the Opening Doors program. Such efforts were extremely labor intensive and 
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had to be repeated every semester. Whether the message did not register or students were too 
busy or simply were not interested, the marketing efforts at these colleges have not yielded the 
numbers of students that were hoped for, and the buildup of research sample groups has pro- 
ceeded more slowly than desired. 

The fact that participation in the Opening Doors study is voluntary raises another im- 
portant limitation. On the face of it, there seems little reason for eligible students not to partici- 
pate in the research — the risks are minimal, and the possible benefits (particularly in sites that 
offer a scholarship to the program group) seem quite large — and yet some students in every 
site have declined the offer. The number of eligible students that chose not to participate at 
Kingsborough appears very small, again because nearly everyone who qualifies for the study 
hears about it during a centralized registration process. By contrast, at the Ohio and Louisiana 
sites, students leam about the study only if they make time to meet with Opening Doors staff or 
to attend an orientation session. It seems likely that the students who attended such a meeting or 
orientation differ from those who did not attend, in terms of motivation, need for services, or 
other characteristics. While the random assignment process ensures that the students who give 
their consent to participate in the study are similar at baseline, it is impossible to know how the 
research sample may differ from students who never attended an Opening Doors orientation or 
who declined to participate in the study. 

In large measure, the limitations of the study reflect the challenges of conducting social 
research in real institutional settings. Recruitment and enrollment for the Opening Doors study 
have to fit into the colleges’ regular enrollment processes. Neither MDRC nor the colleges want to 
create obstacles that will keep students from taking classes or receiving regular services — even if 
it means a less “pure” test than is optimal from a research standpoint. In this regard, Opening 
Doors represents a test case for conducting random assignment studies on community college 
campuses. Just as studies of welfare reform programs in the early 1980s led to more sophisticated 
social experiments in the late 1980s and 1990s, 5 so may Opening Doors yield methodological and 
programmatic lessons that will inform the design of future community college evaluations. 



5 See, for example, Hamilton, 2002. 
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Chapter 4 



The Opening Doors Target Population 
and Research Sample 



From its inception, Opening Doors has been generally concerned with the needs and 
experiences of low-income community college students age 34 and younger, and each of the 
four study sites demonstrates that focus. However, as the individual sites have worked to refine 
their interventions and identify adequate research samples, each has also developed specific eli- 
gibility criteria for Opening Doors that make it unique. For example, Kingsborough targets only 
first-time, incoming freshman who attend school full time; Chaffey concentrates only on stu- 
dents who have been placed on academic probation; the Ohio site excludes continuing students 
who have completed more than 12 credits; and the Louisiana site focuses only on parents. Ap- 
pendix B provides an overview of the criteria for program participation, by site. 

The enrollment of students into the Opening Doors research began in summer 2003 at 
Kingsborough, followed by Lorain in fall 2003, the Louisiana colleges in winter 2004, and 
Owens in summer 2004. Chaffey will begin random assignment in fall 2005. MDRC negotiated 
sample recruitment goals with each college, based on an assessment of the numbers of students 
the colleges could reasonably serve in Opening Doors over a one- or two-year time frame and 
on the numbers needed to detect policy-relevant impacts. For example, a sample of 1,000 stu- 
dents — evenly divided between the program group and the comparison group — would allow 
the evaluation team to determine whether the Opening Doors intervention led to roughly a 2 to 
5 percentage point increase in graduation rates. This “m inim um detectible effect” varies with 
the anticipated graduation rate among comparison group members at each site. A somewhat 
larger sample of 1,400 students would enable the researchers to detect a smaller percentage 
point increase in graduation rates, ranging approximately from 1.5 to 4.0 percentage points. 
MDRC encouraged the colleges to aim for samples of 1 ,400 or more if possible, since this will 
also make it easier to detect effects among subgroups of the sample defined by gender, age, 
prior educational experience, or other characteristics. Kingsborough agreed to a sample goal of 
1,400; the Ohio colleges agreed to a sample goal of at least 1,350; and the Louisiana colleges 
and Chaffey College each agreed to a sample goal of 1,000. 

Table 4. 1 presents descriptive statistics on the research samples for the institutions that 
had begun enrolling sample members at the time that this report was being written. While 
studying these numbers, it should be kept in mind that sample enrollment is still in progress. 
Readers are also reminded that, at each college, half the sample will receive the Opening Doors 



37 




The Opening Doors Demonstration 
Table 4.1 
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Table 4.1 (continued) 
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Table 4.1 (continued) 
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Table 4.1 (continued) 
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Table 4.1 (continued) 
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intervention and half will receive the regular college services. As noted earlier, the random as- 
signment process ensures that there are few or no statistically significant differences in the base- 
line characteristics of program and comparison group members. 



Kingsborough’s Research Sample 

Fitting with the site’s focus on first-time freshmen, the Kingsborough sample is pre- 
dominantly young, unmarried, and without children. As shown in Table 4.1, almost three- 
quarters are financially dependent on their parents. 1 Small minorities of this sample reside in 
households that receive government benefits — such as Temporary Assistance for Needy Fami- 
lies (TANF), food stamps, or Social Security disability benefits — suggesting that this may be a 
low-income population but not one in which most individuals live below poverty. Approxi- 
mately one-third of these students are the first in their family to attend college. 

Because Kingsborough is targeting new freshmen, it is not surprising that the large ma- 
jority of the sample (71 percent) earned a high school diploma or a General Educational Devel- 
opment (GED) certificate within the past year. Half the students who have enrolled to date are 
planning to transfer to a four-year college or university, and more than a quarter (28 percent) are 
working toward an associate’s degree or certificate. Over one-third of the students (36 percent) 
are currently employed, although about two-thirds of these employed students work part time, 
that is, 30 hours or less per week. 

About 30 percent of the Kingsborough sample are not U.S. citizens. Moreover, almost 
40 percent are from families in which both the students and their parents are foreign bom. 
Therefore, it is not surprising that almost half come from homes where a language other than 
English is spoken regularly. These foreign-bom students and their families come from many 
regions of the world, but most commonly they are from Latin America and the Caribbean (2 1 
percent), the Commonwealth of Independent States 2 (10 percent), and Asia (6 percent). 3 How- 
ever, one criterion for program eligibility in Kingsborough is that students not be taking courses 
in English as a Second Language (ESL), so presumably these students are comfortable working 
in English. Finally, Table 4.1 shows that this sample has slightly more women than men and 
that the most prevalent racial/ethnic groups are, in descending order, blacks or African- 
Americans (37 percent), non-Hispanic whites (28 percent), and Hispanics (20 percent). 



Due to rounding, some numbers in the text differ slightly from the corresponding numbers in Table 4.1. 

2 This region comprises Armenia, Azerbaijan, Belarns, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Taji- 
kistan, Turkmenistan, Ukraine, and Uzbekistan. 

3 These statistics reflect baseline survey data that are not presented in Table 4.1. 
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Ohio’s Research Sample 

Table 4. 1 shows that the Ohio sample consists primarily of women (80 percent) who 
have delayed their entrance to college, with many bearing and raising children during their late 
teens and early twenties. The average age is 24.9 years, and 74 percent have at least one child. 
Over 40 percent have two children or more. The average age of the youngest child is 3.4 years. 
A large majority of this sample (76 percent) are unmarried. Only 16 percent completed high 
school or a GED certificate in the past year, and over half (52 percent) did so five or more years 
ago. In contrast to the Kingsborough sample, relatively few sample members in Ohio (14 per- 
cent) are financially dependent on their parents. A majority (56 percent) are currently employed, 
and these are almost evenly split between part- and full-time workers. A large proportion of the 
Ohio sample (52 percent) are planning to work toward an associate’s degree, and more than a 
fifth (22 percent) are working toward transferring to a four-year institution. 

More than half of the Ohio sample are non-Hispanic whites, although there are also sig- 
nificant numbers of blacks or African-Americans (23 percent) and Hispanics (13 percent). Vir- 
tually all are U.S. citizens, and the vast majority (92 percent) are at least the second generation 
of their families to be bom in the United States. Therefore, it is not surprising that, for the large 
majority (91 percent), English is the only language spoken regularly in the home. 

Finally, nearly half the students in the Ohio sample (45 percent) have completed some 
college, but the average number of courses completed is only three, in keeping with the site’s 
requirement that students have not previously completed more than 12 credits. In addition, they 
come from very low-income backgrounds, with significant proportions reporting that they or 
members of their household currently participate in government programs designed for people 
living below the poverty level, in particular the Food Stamp Program (38 percent) and TANF 
(16 percent). Almost a fifth of the sample (19 percent) live in public housing. Many of these 
students (37 percent) are the first person in their family to attend college. 



Louisiana’s Research Sample 

The Louisiana sample that has been recruited to date suggests that the site is reaching its 
targeted population. First and foremost, all the sample members are parents — and are about 
evenly split between those who have a single child and those who have more than one (Table 
4.1). The Louisiana students are almost exclusively female (94 percent), and the large majority 
(81 percent) are unmarried; only 9 percent are married and living with a spouse. In addition, 
they are poor. Fewer than one-third (30 percent) come from a household in which neither they 
nor anyone else participates in at least one of the government programs listed in Table 4.1, and 
more than 1 7 percent live in public housing. Over 40 percent are the first in their family to pur- 
sue a college education, and just about half have a working personal computer at home. 
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Heavily represented in the Louisiana sample are students who have delayed college en- 
try — many of them women who presumably have done so partly in order to bear and rear chil- 
dren. Well over half of them finished high school or a GED certificate at least five years ago. A 
majority (52 percent) are currently employed, and most of those (56 percent) work full time. A 
large proportion of these sample members (56 percent) are planning to work toward an associ- 
ate’s degree. More than a quarter (27 percent) are seeking to complete a certificate program or 
to gain job-specific skills, which reflects in part that one of the Louisiana colleges is primarily a 
technical and vocational institution. 

Finally, the Louisiana sample predominantly comprises students who are either black or 
African-American (85 percent). Most of the remaining sample members are non-Hispanic white 
students (1 1 percent). Nearly all are U.S. citizens, and the vast majority (94 percent) were bom 
in the United States of parents who were also bom here. Similarly, for 93 percent, English is the 
only language spoken regularly in the home. 
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Chapter 5 

The Opening Doors Publications and Dissemination Plan 



The Opening Doors evaluation team will produce a series of documents over the course 
of the demonstration and will adopt multiple strategies to communicate the evaluation’s lessons 
as efficiently and as broadly as possible. Because it takes considerable time to enroll the re- 
search sample and follow up sample members, MDRC will make special efforts along the way 
to share findings in the form of briefings and memos. Targeted audiences include higher educa- 
tion policymakers at the state and federal levels, philanthropic foundations, higher education 
researchers, and professionals from community colleges and workforce development agencies. 
All publications will also be posted on MDRC’s Web site (www.mdrc.org). At present, the fol- 
lowing documents are planned. The projected years of publication are shown in parentheses and 
may change as the study progresses. 

• Building Learning Communities: Early Results from the Opening Doors 
Demonstration at Kingsborough Community College (2005). Kingsborough 
was the first site to begin random assigmnent for the Opening Doors study, 
and it is the first to have impact findings. This report describes the program, 
examines educational outcomes (including grades, course completion rates, 
and semester-to-semester levels of student persistence) for the first cohorts to 
enroll, and presents data from qualitative interviews conducted with a small 
group of students during the first year of the study. 

• Research Briefs on the Ohio Colleges, the Louisiana Colleges, and 
Chaffey College (2005-2006). As soon as sufficient numbers of students 
have enrolled in the study and enough time has elapsed to examine short- 
term academic impacts, the evaluation team will produce research briefs on 
each college’s (or site’s) findings. The briefs will describe the programs and 
present early effects on students’ grades, course completion rates, and semes- 
ter-to-semester persistence. 

• Young Adults’ Engagement in Community College Programs (2006). 

This report will be based on in-depth, one-on-one interviews with 40 students 
between the ages of 1 8 and 25 who are enrolled at Kingsborough and Lorain. 

The focus of the interviews will be on factors that contribute to or impede 
students’ academic progress. Both program and comparison group students 
will be included in the research sample. 
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• Program Implementation and 12-Month Impacts (2006-2007). This re- 
port will use field research and survey data to describe how the Opening 
Doors programs were implemented in each site and what differences in edu- 
cational and other experiences were found between the program and com- 
parison groups. The report will examine impacts on educational outcomes at 
12 months after random assignment and will examine early impacts — if any 
— on students’ employment, peer networks, health, and civic engagement. 

• Program Impacts and Costs at 36 Months (2009). This report will exam- 
ine the longer-term impacts of Opening Doors in the lives of community col- 
lege students and will present findings from the cost study. 

In sum, the Opening Doors demonstration will produce a considerable body of evidence 
on the operations and effects of interventions designed to help community college students 
achieve their academic, employment, and personal goals. The study will also provide insights 
into the opportunities and challenges facing low-income young adults who attend community 
colleges. As the first random assignment test of programs in community colleges, the Opening 
Doors evaluation will set a new standard for research rigor within this sector. MDRC and its 
collaborators welcome advice and support on how to maximize the learning opportunities from 
the study and how to ensure that the evaluation’s lessons help to strengthen educational policies 
and programs for community college students nationwide. 
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Appendix A 

Analytic Methods for Estimating the Impacts 
of Opening Doors 

Cecilia Rouse 
Lashawn Richburg-Hayes 




The primary analytic method to determine program impacts will be comparing average 
outcomes for sample members randomly assigned to the program group or the comparison 
group, using standard statistical tests such as the t-test and chi square test. More formally, the 
plan is to estimate ordinary least squares (OLS) regressions of the form 

E; = a + PRj + £; ( 1 ) 

where E, represents the (eventual) educational attainment of individual i or other 
(eventual) outcomes, such as civic participation 

R; indicates whether the individual was randomly assigned to an Opening Doors 
intervention 

£j is a random error term 

a and (3 are coefficients to be estimated 

The coefficient of interest is (3, as it represents the effect of assigmnent to the Opening 
Doors intervention on the outcome of interest. Because the students are randomly assigned to 
receive the treatment (or not), the background characteristics (including unobserved characteris- 
tics, such as motivation and determination) of the two groups are about the same, on average. 
As a result, ordinary least squares estimation of (3 will provide an unbiased estimate of the “in- 
tent to treat” effect, and it is not necessary to control for other student characteristics. 1 Note that 
the intent to treat estimates the effect, on the outcome in question, of assigning a student to the 
treatment group. While it estimates the gains that a policymaker can realistically expect to ob- 
serve from implementing the program (since one cannot fully control for whether students actu- 
ally take advantage of the enriched services and financial aid), it does not necessarily represent 
the effect of the reforms for those who actually use them. 2 



'The “intent to treat” refers to the mean difference between the outcomes of the program and comparison 
groups in experiments where strict compliance to receiving the treatment (or not receiving it) cannot be ensured 
(Bloom, Forthcoming, 2005). Since participation in Opening Doors is voluntary, one cannot be assured that all 
program group members will receive the intervention. In other words, there may be no-shows, and such no- 
shows reduce the experimental contrast by reducing the difference between the program group and the com- 
parison group with respect to exposure to treatment. Therefore, while the analysis will not be able to answer 
“What is the average effect of the intervention if all members of the target group receive it?” it can answer 
“What is the average effect of making the intervention available to its target group members?” Flowever, in- 
cluding baseline characteristics may reduce the variability of the point estimates, so the analysis will include 
models both with and without such characteristics. 

‘While the measure of the intent to treat is policy-relevant, it is a complex combination of the treatment ef- 
fects for participants and the effects for nonparticipants. Further, it is not the way that most people think about 
program effectiveness. Usually, program effectiveness is thought of in terms of how the treatment affects the 
people who receive it. This more intuitive notion of an impact is often referred to as the average effect of 
“treatment on the treated.” 
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Since the Opening Doors interventions are voluntary, at least some noncompliers can be 
expected. These mainly include program group members who do not receive the intervention 
(no-shows), but they could also include comparison group members who are mistakenly served 
by the program (crossovers). When there are noncompliers, the average effect of the treatment 
on the treated must be estimated through nonexperimental techniques. 3 Two such techniques 
will be used, depending on the nature of noncompliance. If noncompliance is solely due to no- 
shows among the program group, Bloom’s no-show correction technique will be used; it allo- 
cates the difference between the outcomes for program group members and comparison group 
members to the fraction of the program group members who received the treatment. 4 While this 
technique works well for noncompliance due to no-shows, it cannot be used in situations of 
noncompliance due to crossovers. This is because people who receive the treatment may differ 
from those who do not, in ways that are related to the outcomes of interest. 

Therefore, the analysis will also estimate instrumental variables (IV) models in which a 
dummy variable indicating whether the student was randomly assigned to an Opening Doors 
intervention is used as an instrument for truly receiving the Opening Doors services or scholar- 
ships. 5 The random assignment is correlated with actual participation in Opening Doors, but it is 
uncorrelated with the error term in the outcome equation, since it was determined randomly. 
Under plausible assumptions, these models yield consistent estimates of the effect of “treatment 
on the treated.’' In this case, the outcome equation is represented by models such as 

E; = a' + yODi + z\ (2) 

where OD, indicates whether the individual actually received an Opening Doors 
intervention (or the level of intervention) 

Y provides a consistent estimate of the intervention! s) in question on the outcome 

In addition, if assigmnent to the Opening Doors interventions has the effect of increas- 
ing educational attainment among participants (as hypothesized in Equation 1 ; that is, (3 > 0), 
then the causal effect of additional community college credits on student outcomes will be esti- 
mated. Initial placement into an Opening Doors intervention will be used as an instrumental 



3 When there are no-shows or crossovers, one cannot confidently determine which control group members 
are the counterparts of the program group members who receive the treatment. Therefore, nonexperimental 
techniques are necessary. 

4 Bloom, 1984. 

5 This technique — used by Angrist, Imbens, and Rubin (1996) — is called the “local average treatment ef- 
fect.” Using the random assignment to Opening Doors as an instrument for actual treatment receipt solves the 
problem of estimating the average effect of the treatment on the treated when there is noncompliance, because 
randomization leaves the OD; variable uncorrelated with every variable at the time of random assignment, in- 
cluding the error term £. 
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variable for educational attainment. In this case, Equation 1 provides the first-stage equation, 
and the outcome equation of interest is 

Yi = a" + 5E ; + £"i (3) 

where Y; represents the outcome in question (for example, labor market wages, 
health, civic engagement) and the other variables are as defined above 

In this case, the estimate of 5 provides a consistent estimate of the causal effect of addi- 
tional educational attainment on the outcome in question. 
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Appendix B 

Characteristics That Define Students’ Eligibility to 
Participate in Opening Doors, by Community College 




The Opening Doors Demonstration 



Appendix Table B.l 

Characteristics That Define Students’ Eligibility to Participate in Opening Doors, 

by Community College 



Criterion 


Kingsborough 


Lorain County and 
Owens 


Delgado and 
Louisiana 
Technical-West 
Jefferson 


Chaffey College 


Age 


17-34 


18-34 


18-34 


18-34 


Household income 


Not screened 2 


Below 250 percent 
of federal poverty 
level 


Below 200 percent 
of federal poverty 
level 


Below 250 percent 
of federal poverty 
level 15 


Enrollment status 


New or 
continuing 


Only new 
freshmen 


Both 


Both 


Only continuing 


Half time or 


Only full time 


Both 


Both 


Both 


full time 


Other factors 


English as a 
Second Language 
(ESL) students are 
excluded. 


Continuing 
students must not 
have completed 
more than 12 
credits and need to 
show indications 
of academic 
difficulties 
(determined by 
low grades or 
withdrawal from 
courses). 


Students must be a 
parent of at least 
one dependent 
under age 19; must 
have a high school 
diploma or GED 
and pass a college 
entrance exam; 
must not have an 
occupational 
certificate or 
college degree. 


Students must be 
on probation due 
to a grade point 
average below 2.0 
or completing less 
than half of credits 
attempted. 



NOTES: 11 The majority of students who are enrolled at Kingsborough live in low-income households, so 
the Opening Doors study did not require additional income screening. 

b Chaffey’s income criteria for the Opening Doors study are still under discussion. 
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MDRC is a nonprofit, nonpartisan education and social policy research organization. We 
are dedicated to learning what works to improve the well-being of low-income people. 
Through our research and the active communication of our findings, we seek to enhance 
the effectiveness of social policies and programs. MDRC was founded in 1974 and is 
located in New York City and Oakland, California. 

MDRC’s current projects focus on welfare and economic security, education, and 
employment and community initiatives. Complementing our evaluations of a wide range 
of welfare reforms are new studies of supports for the working poor and emerging 
analyses of how programs affect children’s development and their families’ well-being. 
In the field of education, we are testing reforms aimed at improving the performance of 
public schools, especially in urban areas. Finally, our community projects are using 
innovative approaches to increase employment in low-income neighborhoods. 

Our projects are a mix of demonstrations — field tests of promising program models — 
and evaluations of government and community initiatives, and we employ a wide range 
of methods to determine a program’s effects, including large-scale studies, surveys, case 
studies, and ethnographies of individuals and families. We share the findings and lessons 
from our work — including best practices for program operators — with a broad 
audience within the policy and practitioner community, as well as the general public and 
the media. 

Over the past quarter century, MDRC has worked in almost every state, all of the nation’s 
largest cities, and Canada. We conduct our projects in partnership with state and local 
governments, the federal government, public school systems, community organizations, 
and numerous private philanthropies. 




